DOCUMENT RESUME 



ED 315 088 



IR 053 063 



AUTHOR 
TITLE 

PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



Koong, Kai S.; Smith, Harold A. 

Faculty Usage of Higher Education Journals: Toward a 
Taxonomy . 
8 Nov 89 

lOp.; Paper presented at the Annual Meeting of the 
Mid-South Educational Research Association (18th, 
Little Rock, AR, November 8, 1989). 
Viewpoints (120) — Reports - Research/Technical 
(143) — Speeches/Conference Papers (150) 

MF01/PC01 Plus Postage. 

Citation Analysis; *Evaluation Criteria; * Evaluation 
Methods; Faculty Promotion; *Faculty Publishing; 
Higher Education; *Models; -^Scholarly Journals; 
Standards 



ABSTRACT 

A taxonomy and framework for evaluating the quality 
of journals in higher education are proposed in this paper. The 
significance of acquiring and disseminating professional information 
to faculty and administrators in higher education is discussed, and 
it is noted that the journals in which a faculty member publishes are 
sometimes used as critical factors in promotion and tenure decisions. 
Following a review of the literature about hierarchies in higher 
education publishing, a model is presented which offers five 
constructs that affect journal quality: (1) perception, which gauges 
the opinions of selected peers about a journal's quality; (2) 
citations, which measure the number of times a work is cited in 
subsequent research in the area; (3) usage (publishing), a measure 
that shows the number of times fellow educators publish in that 
journal; (4) usage (readership) , identifying how often the source is 
referred to by peers; and (5) factual information, which can be 
obtaine^ from reference publications about journals. A mathematical 
model encompassing flexibility for faculty and academic departments 
with diverse needs is also introduced to help evaluate journals using 
the proposed constructs. The combination of the constructs and method 
are based on the fact that the strength of one can compensate for the 
limitations of the other. A figure illustrates the concept. (14 
references) (SD) 
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Faculty Usage of Higher Education Journals: 
Toward a Taxonomy 

Kai S. Koong and Harold A. Smith 

Need for the Study 

Academic journals are important sources for acquiring and disseminating professional 
information to faculty and administrators in higher education. Journals are especially important 
media for the publication of the findings of their work to add to the body of knowledge. Most 
higher education institutions consider publications produced as critical factors in promotion and 
tenure decisions. However, since many universities have no explicit quantitative 01 qualitative 
publication criteria for promotion or tenure decisions (Gaston, et al., 1975), the identification of a 
body of commonly agreed upon journals in higher education could be valuable to provide some 
direction for faculty and administrators involved in tenure and promotion decisions. There are a 
number of other important reasons for identifying the list of journals in an academic field. For 
example: 

1 . The prestige of a department and the university is often based on the type of journals 
that publish its faculty's research. A department needs to be conscious of the journals which are 
considered prestigious in a particulir discipline, as this can affect the department's success m 
recruiting quality doctoral students and faculty. 

2 . Journal editors also need to know the perceptions of its contributors and readers as to 
the contribution of its publications to the discipline. Such information provides valuable feedback 
to editors as to the impact their publication has in the discipline. 

3 . Faculty members are affected the most because the "publish or perish" syndrome 

requires more than getting one's writing in print The long review and publication lead times 

Presented at the Midsouth Educational Research Association (MSERA) Annual Meeting, 
Little Rock, AR, November 8, 1989. Dr. Koong, Assistant Professor, Information Systems 
Department, Virginia Commonwealth University, may be contacted at Box 4000, 1015 Floyd 
Avenue, Richmond, VA 23284. Dr. Smith, Administrative Intern, Office of Research, Mississippi 
State University, may be contacted at P.O. Box 6343 Mississippi State, MS 39762. 
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for a manuscript to reach print make charting a publication strategy a basic researcher's goal if he 
or she desires to gain a reputation in an area of study. 

Review of the Literature 

Higher education is a broad and diverse field, with increasing numbers of professional 
publications produced. A number ofresearchers have identified journal publishing hierarchies. 
For example, Doke & Luke (1987) found journal publishing hierarchies in perceptions of faculty 
responding to their survey. Perception was used as the mcajiire of quality in this study. 
However, possible Wares in opinions therefore limit the conclusions derived from the data. 
Therefore, this method is flawed because the results are based on opinions instead of empirical 
data. The literature shows several others have responded to the problem (Davis, 1980; Hamilton & 
Ives, 1980; Dezee, 1980; Bayer, 1983; Nelson, et al., 1983; van Over & Ryan; Vogel & Weterbe, 
1984; Gagnon, 1986; Harrington, et al., 1989; King & Trower, 1989). The methods used in some 
of these pioneering investigations have included citation analysis, which measures the impact a 
journal has had within a field by counting the frequency of its appearance in citations by other 
papers/journals in that tield. The assumption is that the quality of a paper is correlated with the 
number of times it is cited in subsequent research. Major problems with this methodology lie in 
the selection of journals, the number of articles in it, the circulation and indexing of the journal, 
and distortions caused by classic articles. 

The respondents in those earlier studies were limited to a list of journals selected by the 
respective researchers, thus possibly introducing biases into the findings. This could be a 
particularly serious limitation in a field as broad as higher education. Such problems and 
limitations of journal assessment methods were found to be true in the field of computer 
information systems. Koong and Weistroffer (1989) conducted a survey using an open-ended 
questionnaire and made some interesting observations. For example: 
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1 . Perception is a good predictor of reading habits, but a poor predictor of publishing 
habits. Where a researcher publishes does not necessarily reflect his or her source for identifying 
reading material. 

2 . Certain journals rank high for acquiring knowledge but not for disseminating 
knowledge. Apparently, trade journals are important sources of information acquisition. 

3 . There are certain key journals for a particular discipline irrespective of methods used for 
assessing their quality. In the computer information systems field, the two unchallenged high 
ranking journals are Management Information Systems Quarterly and Communications of 
Association of Computing Machinery. 

The profile results of Koong and Weistroffer's study supported the notion that the survey 
respondents were from a normally distributed population and were representative of the 
involvement of a typical faculty member in an American Association of Colleges of Schools of 
Business accredited university. A study in progress on higher education journals is expected to 
have similar findings. 

These studies have important implications for researchers in higher education. The major 
objective of this paper is to propose a framework to help identify journals that are relevant for 
educators in the field of higher education. A secondary objective is to suggest methods that can 
help educators perform a robust evaluation of the journals publishing manuscripts in the area of 
higher education. 

This paper does not refuts or make obsolete earlier research, but does point out a way of 
augmenting and extending eavlier findings on professional journal usage. Faculty should know 
which journals are widely respected for disseminating or obtaining information in order to target 
their own publication efforts. Department administrators should know which journals are 
respected for obtaining knowledge in order to evaluate faculty publishing success. Furthermore, 
identification of the journals used in any field may be crucial to the efficient allocation of limited 
library resources. Therefore, faculty and administrators in higher education, as well as librarians, 
are likely to benefit from results of this methodological extension. 
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Toward a Taxonomy 

Various assessment methods have been used to measure journal quality. Each method has its 
advantages and disadvantages. One method cannot determine the total contribution or impact a 
journal has in its discipline. However, an eclectic assessment is possible by combining the 
strengths of each method. Such a methodology can be used to assess the total quality of a journal. 
A taxonomy of such a method is proposed in Figure 1. 

As indicated in Figure 1, the proposed conceptual framework for assessing journal quality 
will consist of five constructs. Each construct can assess the impact of a journal from a different 
dimension. Furthermore, these five constructs can enable researchers to evaluate a journal and 
develop their publishing strategy to fit their respective needs. For example: 

Perception: This construct measures the opinion of selected peers about the quality of a 
journal. In most cases, the people who are included in this type of a measure are leaders in the 
field. This measure will be useful to senior faculty or those in universities who value peer 
evaluations of their performance. 

Citation: This method measures the number of rimes a work is cited in subsequent research 
in the area. This measure may be valuable for leading educators in the field in measuring the actual 
need or value of the research attempted. 

Usage - Publishing: This construct measures the usefulness of a journal as gauged by the 
number of times fellow educators use the journal for disseminating knowledge. Fresh doctorates 
may find this measure helpful because it is an indicator that one is publishing in the "right" 
journals. 

Usage - Readership: This construct also measures the usefulness of a journal. However, 
the impact is gauged by the number of times it is used by peers for acquiring knowledge. The 
interesting phenomenon about this measure is that the manuscripts are written primarily for 
practitioners rather than academicians. However, for faculty of universities that emphasize a more 
"hands on" approach, this can be a valuable method. 

ERIC 4 6 
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Factual information: This measure uses key variables that are publicly available. The 
Cabell Directory is one such publication that contains information on journals such as size of 
circulation, lead times, acceptance rates, and targeted population. This information can be a 
valuable indicator of she impact a journal has in the field. 

In addition to the five constructs, it is important to note that flexibility of assessment 
should be introduced into the taxonomy. Different universities and faculties do not place equal 
value on the constructs. One such proposal is to use a multi-criteria approach toward building the 
index for evaluating journals. As with all criteria decision-making models, the weights can be 
tailored to the needs of the departments themselves or an individual faculty member. An example 
of such a model is presented below: 

Ij = W1C1X1 + W2C2X2 + W3C3X3 + W4C4X4 + W5C5X5 + Ej 

where: 

Ij - Index 

Wi -- Weights of variables/assessment methods 

O -- Coefficient of variables/may be positive or negative 

Xi - Assessment methods (perception, citation, usefulness - publishing, usefulness - 
readership, and factual information respectively) 
Ej - Randomized error of the model 
and Wi£0 

Ij£ 1 andlj>0 

A department can easily control the weights and have different models for evaluating faculty 
publishing for different ranks. For example, using the model above, junior faculty can be assigned 
higher weights on the latter variables because the larger quantity produced (at the expense of 
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perceived quality) can help them gain tenure more feasibly. On the other hand, for more senior 
faculty, the weights of the earlier variables can be higher because these faculty members have an 
established record and should concentrate on building a higher perceived reputation in the field to 
attain promotion requirements for higher ranks. 

Conclusion 

The objective of this paper is to present a taxonomy and framework for evaluating journal 
quality in higher education. A model presenting five constructs that affect journal quality was 
presented. A mathematical model encompassing flexibility for faculty and departments with 
diverse needs was also presented to help evaluate journals using the constructs proposed. The 
combination of constructs and method are based on the fact that the strength of one can compensate 
for the limitations of another. Such an eclectic method, hopefully, may provide for a more rational 
and empirical method to evaluate faculty publishing success. 
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